Papers with pragmatic reasoning
Learning from Omission (P19-1)
Copied to clipboard
| Challenge: | a recent study shows that pragmatic reasoning improves language understanding systems . end-to-end training produces more accurate utterance interpretation models . |
| Approach: | They show that pragmatic reasoning can improve the quality of learned meanings . they draw pragmatic inferences from listening to what a speaker says and not to what they do . |
| Outcome: | The proposed model improves the quality of language understanding models when sparse data is used. |
He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmatics (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing models for factuality prediction are lacking for English . Traditionally, event factualism is triggered by fixed properties of lexical items . |
| Approach: | They propose a model that exploits common surface patterns that correlate with factuality labels. |
| Outcome: | The proposed model achieves the best performance on four factuality datasets. |
Investigating the Performance of Transformer-Based NLI Models on Presuppositional Inferences (2022.coling-1)
Copied to clipboard
| Challenge: | Presuppositions are assumptions that are taken for granted by an utterance. |
| Approach: | They propose to use heuristics to create alternative "contrastive" test cases . they also analyze samples from ImpPres datasets to better understand their predictions . |
| Outcome: | The proposed model performs better on the ImpPres dataset than on the other datasets. |
GRICE: A Grammar-based Dataset for Recovering Implicature and Conversational rEasoning (2021.findings-acl)
Copied to clipboard
| Challenge: | a grammar-based dialogue dataset, GRICE, is designed to bring implicature into pragmatic reasoning in conversations . implicature recovery is a key component of open-ended dialogue reasoning . |
| Approach: | They propose a grammar-based dialogue dataset to bring implicature into pragmatic reasoning . they use a hierarchical grammar model to generate the entire dataset . |
| Outcome: | The proposed model shows a significant performance gap between baseline methods and human models . the model shows an overall performance boost in conversational reasoning . |
PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations (2023.findings-acl)
Copied to clipboard
| Challenge: | Mars? - PragmatiCQA |
| Approach: | Mars? - The Paper . |
| Outcome: | The proposed dataset features 6873 QA pairs that explores pragmatic reasoning in conversations over a diverse set of topics. |
Modeling Cross-Cultural Pragmatic Inference with Codenames Duet (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on pragmatic reasoning tests using simple word reference games with unidentified speakers and listeners, but speakers' sociocultural background shapes their pragmatic assumptions. |
| Approach: | They propose a dataset which operationalizes sociocultural pragmatic inference in a word reference game. |
| Outcome: | The proposed model improves clue-giving and guessing tasks by accounting for background characteristics and the game context. |
Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition (2020.acl-main)
Copied to clipboard
| Challenge: | Natural language inference (NLI) is an increasingly important task for natural language understanding . however, the ability of NLI models to make pragmatic inferences remains understudied . |
| Approach: | They use semi-automatically generated sentence pairs to evaluate whether NLI models make pragmatic inferences. |
| Outcome: | The proposed model trains on multiNLI and shows that it learns to draw pragmatic inferences. |
Born Pragmatic, Trained to Hallucinate? Quantifying the Origins of Contextual Bias in LLMs via the PaCE Benchmark (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models excel at capturing communicative intent, but they have a side effect: pragmatic hallucination. |
| Approach: | They propose a benchmark to quantify the impact of pragmatic hallucination on large language models . they propose RLHF and SFT to induce a strong tendency for pragmatic over-attribution . |
| Outcome: | The proposed model outperforms existing models in predicting pragmatic hallucinations . the evaluations show that current alignment paradigms lack precise control over pragmatic boundaries . |
Collaborative Rational Speech Act: Pragmatic Reasoning for Multi-Turn Dialog (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing extensions of Rational Speech Act face challenges in scaling to multi-turn, collaborative scenarios. |
| Approach: | They propose a Rational Speech Act extension that optimizes a gain function adapted from rate-distortion theory to model multi-turn dialog by optimizing a model gain . they demonstrate the effectiveness of CRSA on referential games and template-based doctor–patient dialogs in the medical domain. |
| Outcome: | The proposed model yields more consistent, interpretable, and collaborative behavior than baselines, paving the way for more pragmatic and socially aware language agents. |
Communicating with Speakers and Listeners of Different Pragmatic Levels (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using a model of the speaker's intentions, people can achieve pragmatic interpretations using a variety of reasoning abilities. |
| Approach: | They propose to model the interaction between speakers and listeners with different levels of pragmatic competence and to model their level of reasoning abilities. |
| Outcome: | The proposed model is based on a simulating language learning and conversing between speakers and listeners with different levels of reasoning abilities. |
Rethinking Pragmatics in Large Language Models: Towards Open-Ended Evaluation and Preference Tuning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to assess social-pragmatic inference in large language models are inadequacy, and preferential tuning is the best approach. |
| Approach: | They propose to use free-form models' responses as a measure to assess social-pragmatic reasoning and advocate for preference optimization over supervised finetuning (SFT). |
| Outcome: | The proposed model outperforms supervised finetuning (SFT) and offers a near-free launch in pragmatic abilities without compromising general capabilities. |
LLMs in Sarcasm Detection? It’s elementary! (Or is it?) (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are often cited for their sophisticated pragmatic reasoning, but they collapse to random guessing on organic human speech. |
| Approach: | They propose that LLMs have near-human competence in sarcasm detection . authors propose that this proficiency may be deceptive . |
| Outcome: | The proposed model performance on synthetic leaderboards is a statistical mirage of competence. |
Read the Room, Read the Image: Understanding Indirect Speech Acts in Multimodal Visual Contexts (2026.findings-acl)
Copied to clipboard
Jaehee Kim, Ji Hoon Chung, Seoyoon Park, Unsol Kim, Kyungwon Park, JiHak Kim, Yi-Jun Chen, Hansaem Kim
| Challenge: | Existing benchmarks focus on explicit context, but do not address context-dependent pragmatic understanding. |
| Approach: | They propose a benchmark for evaluating ISA understanding through integrated reasoning over visual context and dialogue. |
| Outcome: | Experiments show that state-of-the-art models struggle with visually grounded indirect speech acts . linguistic meaning emerges through the relationship between an utterance and situational context . |
DRInQ: Evaluating Conversational Implicature with Controlled Context Variation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent large language models exhibit strong conversational fluency but are unreliable when interpretation depends on reasoning that integrates social and contextual cues. |
| Approach: | They propose a semi-automated pipeline that produces question-context-interpretation instances with systematic variation to isolate pragmatic variation while holding each question’s surface form fixed. |
| Outcome: | The proposed framework isolates pragmatic variation while holding each question’s surface form fixed. |
Language Models Identify Ambiguities and Exploit Loopholes (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that exploit loopholes identify and reason about ambiguity and conflicting goals, presenting a potential safety risk. |
| Approach: | They propose to study the responses of large language models to loopholes by examining ambiguity and pragmatics in LLMs. |
| Outcome: | The proposed models can identify ambiguities and exploit loopholes to satisfy their given goals as opposed to the goals of the user. |